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Abstract 



In this paper we study the behavior of a continuous time random walk (CTRW) on a stationary 
and ergodic time varying dynamic graph. We establish conditions under which the CTRW is a 

stationary and ergodic process. In general, the stationary distribution of the walker depends on the 
walker rate and is difficult to characterize. However, we characterize the stationary distribution in 
the following cases: i) the walker rate is significantly larger or smaller than the rate in which the 
graph changes (time-scale separation), ii) the walker rate is proportional to the degree of the node 
that it resides on (coupled dynamics), and iii) the degrees of node belonging to the same connected 
component are identical (structural constraints) . We provide examples that illustrate our theoretical 
findings. 
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1 Introduction 



During the last decade, there has been a wide interest in characterizing and modehng the structure of 
various networks, from neural networks, to the web, to Facebook friends. Real networks are inherently 
dynamic in the sense that both nodes and edges come and go over some time-scale. However, most efforts 
consider the network as either a single static graph or as a pre-defined sequence of graph configurations. 

Random walks are an important building blocks for characterizing networks. Their simple behavior on 
static networks has been explored to devise algorithms for various purposes, from ranking to searching 
(details in Section [5|. However, very little is known about the long-term behavior of random walks on 
dynamic networks. 

In this paper, we study continuous time random walks (CTRWs) on stationary and ergodic dynamic 
graphs. We make the following contributions towards this goal: 

• We consider stationary and ergodic dynamic graphs where nodes are always present in the network 
but edges are allowed to come and go over time, including the cases where the network consists 
of several connected components. We introduce the notion of T-connectivity and show that if 
the dynamic graph is stationary, ergodic and T-connected then the CTRW is also stationary and 
ergodic. In the full generality of our framework, the stationary distribution of the walker depends 
on the walker rate and is difficult to characterize. However, 

• we characterize the stationary distribution of the random walk for several cases: (i) Time-scale 
separation: the walker rate is significantly larger or smaller than the rate in which the graph 
changes; (ii) Coupled dynamics: the walker rate is proportional to the degree of the node that it 
resides on; (Hi) Structural constraints: the degrees of nodes within any connected components 
are identical (but can vary among different components). 

• We evaluate numerically several examples to support our theoretical results and illustrate their 
applicability. We also present a simple illustrative DTN application. 

The remainder of this paper is organized as follows. Section [2] presents the proposed modeling framework 
together with some definitions and properties. Section [3] presents the stationary distribution of CTRW 
when the walker rate (for being too fast or too slow in respect to the speed the graph changes) allows 
a time scale decomposition of the combined walker and graph processes; we also present conditions 
under which the CTRW stationary distribution is invariant under time scale changes. Section [4] presents 
the numerical examples and applications. In Section [5] we discuss the related work. Finally, Section |6] 
concludes the paper. 
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2 Model formulation 



In this section we define important concepts that will be used throughout this paper. We define a dynamic 
graph as a simple marked point process. A continuous time random walk (CTRW) over a dynamic graph 
is also a marked point process. We define the concept of T-connectivity to express the ability of nodes 
to be connected in time. We also define conditions for stationarity of the graph process and the CTRW 
process. We start by defining the graph dynamics. 

Definition 2.1 (Dynamic Graph) The time evolution of the graph under consideration is given by 
the (possibly simultaneous) addition and deletion of edges. Let V denote a finite set of n nodes and 
let A denote a finite set of m adjacency matrices A = {Ak}^^i, where Ak is an n x n unweighted 
symmetric adjacency matrix. A dynamic graph is a simple Random Marked Point Process (RMPP) 
4* = {(^i, 'S'i)}ieZ7 where Z is the set of all integers, Xi £ A denotes the i-th graph configuration and Si 
is the time that the network spends in that configuration. 

Because is simple, P{Si = 0) = for all i. Moreover, we assume < E[Si] < oo, z g Z. We use 
Gk to denote the graph configuration that has adjacency matrix Ak, k = l,...,m. Throughout this 
work we use A^. and interchangeably. To simplify our analysis we focus on unweighted adjacency 
matrices. However, edge weights can be easily accounted for in the walker rates and thus our results are 
also applicable to weighted dynamic graphs. We define the process {A{t)}teM, as 

A{t) = Y,xaiT,<t<T.^,}, (1) 

where by convention ■ ■ ■ < T^i < Tq < < Ti < ■ ■ ■ are the successive times at which the graph switches 
to another configuration with Si :— T^+i — Ti,i£ Z, i.e., A{t) denotes the adjacency matrix of the graph 
at time t; A{t) is right-continuous. 

Assumption 2.1 The graph process Vf* is stationary and ergodic. 



Throughout this paper we assume that Assumption |2 . 1 1 holds . The following proposition shows that the 
ergodicity and stationarity of = {Xi, Si}i^z implies {A(i)}tgR is also stationary and ergodic. 



Proposition 2.1 Under Assumption 2.1. {A(t)}te]gL is stationary and ergodic. 



A proof of Proposition |2.1| is provided in Appendix |X] In what follows we provide a result that will be 
useful in our proofs: 



3 



Theorem 2.1 Under Assumption \2.1\ 

lim \ I l{A(x)=Ak}dx = CTfc , P-a.s., (2) 



1 



where > Q. 



Proof. The proof of Theorem |2 . 1 1 follows directly from Proposition 2.1 and Birkhoff's ergodic theorem 
(Shiryaev [H pg. 409, Theorem 1]). □ 



We refer to cr = (cri, . . . , am) as the stationary distribution of {A{t)}t>Q- We note in passing that a direct 
consequence of Theorem 2.1 is that limf_j.oo \ Jq ^{A(x)=Ak}'ix = E[l^Xi=Ak}Si]/ E[Si] — Ok, for all z, k. 



Another important property of a stationary and ergodic ^' that we use throughout this work is the 
uniform convergence of the time average regardless of the initial conditions. 



Proposition 2.2 (Uniform mixing) Under Assumption 2.1 



lim - 



P{A{x) = Ak\B)dx ^ ffk , fc=l,...,TO, 



(3) 



for any Borel set B such that P{B) > 0. Furthermore, the convergence is uniform with respect to any B. 



Proof. A short proof is presented. An extended proof can be found at Appendix |B] Stationarity 
and ergodicity imply Integrating both sides of ([2| w.r.t. dPiuj) for w e S and using the bounded 
convergence theorem, Fubini's theorem and Egorov's theorem |10> pg. 43, Theorem 2] gives ([S]) uniformly 
in B for any Borel set B such that P{B) > 0. □ 

It is possible for one or more graph configurations to consist of two or more disconnected components, 
in which case we have to be concerned about whether a walker can move from any node to any other 
node over time. In what follows we introduce the concept of connectivity over time between two nodes 
(denoted as T-connectivity). 

Definition 2.2 (T-connectivity) Two nodes u and v are said to he T-connected in 5', if they are 
connected in the graph induced by adjacency matrix A — \l kA^, where V is the binary OR operator. 

An alternative equivalent definition of T-connectivity uses A = X^feLi ^fe in place of A = W^Ak. T- 
connectivity can also be stated as a graph property. 

Definition 2.3 (T-connected graph) A dynamic graph is said to be T-connected if all pairs of nodes 
are T-connected. 
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We now define a Continuous Time Random Walk (CTRW) over the dynamic graph, starting at time 
t = 0. 



Definition 2.4 (CTRW) A continuous time random walk (CTRW) on a dynamic graph {A{t)}t^B, 
associated with RMPP ^, is a process {{A{t),U{t))}t>o, where U{t) G V is the position (node) of the 
walker at time t. The times between CTRW steps are independent and exponentially distributed. The 
rate at which the walker makes a step at node U{t) = v when A{t) = Ak is Jk,v At the time the walker 
leaves v, it chooses one of its currently connected neighbors in Aj^ (if any) uniformly at random. When 
V has no neighbors in Ak the walker stays at v until the next step event. 



Let r = {7fc,v} denote the set of walker rates associated with {A{t)}t>o. We will find it useful to express 
jk,v as jk,v = Pk.vl, k = 1, . . . ,TO and w G V. Walker rates of interest to us include pk,v = 1 (denoted 
CTRW with constant walker rate) and [3k,v ~ dk.v, where dk,v is the degree of v given adjacency matrix 
Ak (denoted CTRW with degree dependent walker rate). 

The above framework is general enough to describe several more particular dynamic graph models, such 
as renewal processes and Markovian processes. In a Markovian process Si is exponentially distributed 
and P[Xi = Xi\Xi_i = Xi_i,Xi_2 = Xi-2, • • •] = P[Xi = Xi\X^-i = Xi G i = 0, 1, 



Notation Summary 



= {{Xi, Si)}i^z dynamic graph process (in events) 
{A{t)}t,£B. dynamic graph process (in time) 

{iA(t),Uit))}t>o CTRW process 
a = (ci, . . . , am) stationary distribution of {A(t)}t>o 
lk,v = (3k,vl walker rate of the CTRW at node 

V at configuration A). 



3 Characterizing RWs: Stationary Behavior 

In this section we focus on the stationary behavior of a RW on a dynamic graph, in particular the steady 
state fraction of time the walker spends in each node of the network: tt = (tti, . . . , 7r„). 

The steady state distribution tt is trivial if the graph is static and T-connected, i.e., A{t) = A', Mt > 0, 
where A' is a symmetric (0, l)-adjacency matrix of a connected graph. The stationary distribution is 
unique and given by 

""-^ T d h ' 
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where dy denotes the degree of node v and is the walker rate at node v. Characterization on a dynamic 
graph is much more challenging. How can we characterize tt on dynamic graphs? When does tt converge 
for dynamic graphs? When can we give an expression for tt? We focus on three cases where we are able 
to characterize this behavior. 

We begin our study of the stationary behavior of the process {U{t)} with the following. 

Theorem 3.1 IftheRMPP'^ is T-connected, stationary, and ergodic, then the process {{A{t),U(t))}t>o 
is stationary, and the stationary distribution is unique. 

Proof. We create a new marked point process, {{X'^, S'^)}^ that is a superposition of the graph process 
{{Xj, Sj)}"^ and a Poisson process having rate 7,nax = max^gr J' (recall that F is the set of walker 
rates). To simplify our proof we shall assume P[Tq = 0] = 1, where Tq is as defined in ([!]), although 
the proof is valid for any To < 0. We associate the mark "0" with each point of the Poisson process, 
Hence X'^ e {0} U {Ai, . . . , A„i}. As both the graph and Poisson processes are event stationary, the new 
merged process is also event stationary Section 1.3.5]. Let t'^ < t[ < ■ ■ ■ < t[ < ■ ■ ■ denote the times 
associated with this new process, {{X^, Sl)}"^ . Consider the process {{A'^,Ui)}^ where Ui denotes the 
walker position at time t[ and A'^ denotes the adjacency matrix during the period [t'i,t[^^). Note that 
{A'^, Ui) ^ A X V takes values from a finite set. {{A'^, Ui)} is described by a stochastic recursion of the 
form {A'^,Ui) ^ (t){Ui-i,X[,Ri) where 

A\ = MuLi,K R^) = i{xl = o}a:_i + 1{XI + Q}Xl 

for all i = 0, . . .. Here X^, Si are as previously defined and {Ri} is an iid sequence of uniformly distributed 
rvs in [0, 1] independent of {{X'^, S^)}. These auxiliary rvs are used to choose the neighbor to which the 
walker goes or to remain stationed at its current node. Note that {{X'^, S-, -Ri)} is stationary. 0b is defined 
so that when X'^ ^ 0, the walker does not move {Ui — Ui-i) but the graph changes to configuration X^. 
If X'^ = 0, the walker moves from Ui with probability 7x',c/i/7max, moving to one of its neighbors (in 
configuration A'^_i) chosen uniformly at random (using Ri). 

Theorem 1 in [12] states that if there exists a random subset, B C A x V such that a sample path 
monotonicity condition ((5) in jl2j ) holds and the existence of a finite non-empty sample path absorbing 
set ((9) in [T2) exists, then it is possible to construct process S'-, A'^, Ui} that is event stationary as 
is {Ui}. In our case, because our state space is finite, these conditions trivially hold by taking B ~ AxV. 
Since {X^, S'^, A'^, Ui} is event stationary, {A'{t), U (t)} is also stationary. It follows from our construction 
that {A{t)} ^ {A'{t)}; hence {A{t),U{t)} is also stationary. 

We now address the question of uniqueness through a coupling argument. Consider two random walks 
{{Ui{t)}t>Q and {U2{t)}}t>o that differ in their starting locations at time t ~ 0, Ui{0) = ui and U2{Q) — 
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U2- We are interested in establishing that the time, T at which they meet, is finite a.s.. After time T 
the processes couple, i.e., for t > T, Ui{t) = U2{t). This is possible because the times between steps 
are exponentially distributed random variables. Thus, when T is finite, the above coupling argument 
implies that {Ui{t)} and {U2{t)}, which we have shown to be time asymptotic stationary, have the same 
stationary distribution. 

It is left to show is that T is finite. We sketch the argument here and relegate the details to Appendix [Cj 
The basic idea is to identify intervals of time of length T' < oo starting at times iT' > 0, i = 0, 1, . . ., and 
based on the ergodicity and time stationarity of the graph process to establish a lower bound, po, on the 
probability of two walkers coupling during interval [iT' , {i + 1)T']. The probability that the walkers do 
not couple within the interval [0, jTg) is upper bounded by (1 — Po)"'- Thus the walkers couple in finite 
time a.s.. □ 

Note that if the graph process is not T-connected, then it is possible for the system to exhibit multiple 
stationary regimes that depend on the initial position of the walker. Next we characterize tt when there 
is a time-scale separation of the walker and graph dynamics. 



3.1 Stationary Behavior under Time-scale Separation 

Consider a scenario where the walker is either much faster or much slower relative to the rate that the 
graph changes configurations. In this case, we have a time-scale separation between the two processes 
that allows us to characterize tt. 



3.1.1 The Fast Walker 

Let us first assume that the walker rate is much larger than the rate at which the graph changes. For 
a sufficiently large 7, the steady state probabilities of the random walk tt is a linear combination of the 



corresponding probabilities of the adjacency matrices Ai, . . . , Am. Theorem 3.2 formalizes this argument 
for the case that every adjacency matrix in A is connected. We will describe, under certain conditions, 
how to relax this assumption later. 

In preparation, let ^k,v = Pk.vl and let 7r('^)(7) = (7rJ'^^(7), . . . ,7ri'^^(7)) denote the steady state distribu- 
tion of a random walk on the undirected graph with adjacency matrix as a function of 7 > 0. It is 
given by 

7rW(7)EE7r(^) = , v&V-k = \,...,m. (4) 



independent of 7. Let 



J2jevdk,j/I3k,j ' 
ttC^) (7, t, w) = {l,t,w),..., (7, t, w)) 
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denote the distribution of the CTRW on Ak at time t > starting from node w. Assume Ak is irreducible, 
k — 1, . . . ,m, i.e., the graph with adjacency matrix A^ is connected. Because the random walk with 
adjacency matrix Ak is described by a time-reversible Markov chain, 7r('^)(7, w) can be expressed as 

n 

7rW(7,t,u;) =7rW +^c;A2e^'=^-^*, w & V, t > , (5) 

where — Xki > \k2 > ••• > ^km are the eigenvalues associated with Qk{l)/l where Qk{l) is the 
infinitesimal generator associated with the random walk with parameter 7 on the graph with adjacency 
matrix A^, and {cj'^^} are vectors related to the j-th eigenvector of the random walk and the initial 
condition that the walker begins at node w. 



Theorem 3.2 // the graph process '5 is T-connected, stationary, ergodic, and the configurations are 
always connected, then in the limit as 7 00, the stationary distribution tt of the random walk is given 
by 

711 

7r = ^afe7rW. (6) 



fc=i 



Proof. We show that the walker steady state distribution 77(7) — >■ tt as 7 — ^ cjo where tt is given in ([6]). 

We focus on the i-th graph configuration, Xi, i>0. To simplify our proof we shall assume P[To = 0] = 1, 
where Tq is as defined in Q. Let fI^\x) — P{Si < x\Xi — Ak) and define 17^*^'' (7) to be the stationary 
distribution of the CTRW while the graph is in state Xi — Ak- Let Pi.wil) denote the initial walker 
distribution when the process first enters graph configuration Xi . rif^ (7) is defined as 



/•OO -1 />t 



POO 1 nt ^ 

where the second equality follows from ([5]). We focus on the second term, henceforth denoted as C^, 
which we show goes to zero as 7 — ?> 00. We focus first on the singularity of the 1/t term due to the first 
integral starting from zero, 

w^V ' J— 2 

where |c| is the vector whose components are the absolute values of the components of c and e is a vector 
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of all ones. Evaluating the second integral and recognizing that 1/i < for t > 7-1/4 yigijg 



w^V J— 2 I 



Both terms go to zero as 7 — > cx). Consequently — > and 

lim 77^^(7) = tt'*^', Vfc 

7— foo 

This holds for all i; therefore it holds when the graph is in steady state and removal of the conditioning 
on the graph configuration yields □ 

We now focus on the case where one or more of the graph configurations consists of disconnected com- 
ponents. We relabel the nodes in each graph configuration in order to easily identify the disconnected 
components. For each of the original m adjacency matrices, A^., we rearrange the n nodes into subsets 
of connected components. In other words, consider graph configuration Gk associated with adjacency 
matrix Ak- Partition the set of nodes in Gk into Ok sets, each containing only connected nodes. The 
sets correspond to Ok adjacency matrices, {Ak.i, . . . , Ak.ok} ^nd graph configurations {Gk,i, • . • , Gk.ok}- 
Let Vk.i denote the set of nodes in configuration Gk^i- 

Let i/'(7) = ("01,1(7)1 V'm, 0^(7)) denote the vector of stationary probabilities that the walker is in 
the different components of all of the configurations when the rate parameter is 7. Because the CTRW 
process is ergodic, this vector exists and is given by 

1 /■* 

t yoo Xi In 



Define 



= lim ■j/)(7) . (7) 

7— foo 

We will describe conditions under which can be computed later. In what follows we show that if xp 
exists then we obtain the stationary distribution tt of the random walk in the limit as 7 — )■ 00. 

Let TT*^*^''^ be the steady state distribution of a random walk on the undirected graph with adjacency 
matrix A^^i- Similar to equation HaH 



^If the denominator is zero (i.e., there are isolated nodes) we simplify our notation assuming the ratio 0/0 = 1. 
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and k = 1, . . . ,m,l = 1, . . . , Ofe. 



We let TT^'^^ be the concatenation of vectors tt^'^''^ that is, tt^'^^ = (tt^*^'^^ 

Note that vectors tt^*^-* for all 1 < k < m have the same cardinality. 
We have the following result. 

Theorem 3.3 // the graph process ^ is T-connected, stationary, and ergodic, and ip exists, then in the 
limit as 'y oo, the stationary distribution tt of the random walk when graph configurations may be 
disconnected is given by 

m 

7r = ^7r('=). (8) 
Proof. The proof is similar to that for the case where all graphs are connected. □ 



7r(fe.ofc)) and 



In general ip is difficult to compute. The difficulty here lies in that the walker state at time to can now 
depend on {A(t)}Q°, something that was not possible when all configurations were connected. However, 
tp is easily characterized when the underlying transitions between configurations are described by a 
Markov chain and the times that the graph remain in a configuration correspond to mutually independent 
sequences of iid random variables; one sequence for each configuration. Let P = [pij] denote the m x m 
transition probability matrix for the graph configurations at the time of transitions between graphs and, 
with an abuse of notation, let {Sk,i}o^, k = 1, . . .m, denote the mutually independent iid sequences of 
configuration holding times for the graph configurations. 

We focus now on transitions that the walker makes between connected components in two different 
graph configurations, say the ji-th connected component in configuration Gi~i and the j2-th connected 
component in configuration Gfc^- We define a transition probability matrix P = \pki,ji;k2,j2] ^ follows 

Pk ■ k ■ =Pk k ^-^^^l-^l^^^^.^-^ dk,,y/Pk,,V 

yki,3i;k2,32 yki,k2 sr-^ rl I R ' ^ ' 

^weVki,ji "'ki,w/ Pki,w 

The first term accounts for transitions between graph configurations and the second term accounts for 

the walker dynamics. Here P can be thought of as the transition probability matrix for a discrete time 
Markov chain that characterizes the; subgraphs visited by a random walk at graph transitions in the limit 
as 7 — >■ oo. This chain is irreducible provided the graph is T-connected. Let ip* — {ipl^i, . . . 
denote the stationary distribution of this MC. The earlier introduced probability distribution tp can be 
expressed in terms of tp* as follows 

'lpk,l = v^m v^Oi 1^ FFCI ' AkeA]l^l,...,Ok, (10) 
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Note that the above characterization depends on the independence and identical distribution assumptions 
of the configuration holding times. This, along with (|8| fully characterizes the stationary distribution of 
the walker in the fast walker regime for the Markovian environment. 

3.1.2 The Slow Walker 

In this section we consider the walker stationary distribution, tt, at the other timescale decomposition, 
namely where the graph dynamics speed up relative to the walker. Consider a walker with the set of walker 
rates F 
walking a dynamic graph 5". We consider the RMPP ^E''^"' = {{Xi, aSi)}i£2, that is a speed up of 
the RMPP ^1/ by a factor of a, < a < 1, and characterize the CTRW on ^f^") as a — > 0. We denote 
by A(°'(t) the state at time t of the dynamic graph corresponding to the RMMP We do this in 

two steps. We first consider an observer of who makes observations according to a renewal process 
with the property that it has a continuous non-increasing probability density function with finite mean. 
We then determine conditions under which the observer is guaranteed to observe independent instances 
of the graph with probability given by the stationary distribution of the graph. Finally, we consider a 
Poisson observer and couple the walker with it in order to characterize the stationary distribution of the 
walker. 

We introduce a renewal process {Wj}j" where Wj denotes the time between the (j — l)-th and j-th 
observations with CDF G{x) (with PDF g{x)) satisfying the following assumption. 

Assumption 3.1 The pdf g{x) := dP{Wi < x)/dx is differentiable with g'(x) = dg{x)/dx, non- 
increasing, nonnegative, with (i) g{Q) < oo, (ii) g{x)dx = 1 and (Hi) E\Wj\ = xg{x)dx :— D < oo. 
As a consequence of the previous assumptions (iv) xg'{x)dx = —1 (Hint: use an integration by part 
and note that lim^^oo xg(x) — thanks to (Hi)). 

Assumption |3 . 1 1 holds if Wi is exponentially distributed with parameter 7 < oo. It also holds if Wi has 
a Pareto distribution with Pareto index strictly larger than one. We will now observe the graph at the 
renewal instants X]fe=i ^fc' J ^ 1- 

We denote by A^°-\t) the state at time t G M of the graph associated with the RMPP ^J/^"), namely. 



so that = A{t/a) for all t e M, where T, is as defined in 0. We denote by = A^^^Wi + 

■ ■ ■ -\-Wj) the graph configuration of A^'^\t) at the j-th renewal instant (j > 1). 



00 




(11) 
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Lemma 3.1 Let ^'^"^ he the RMPP associated with the stationary and ergodic dynamic graph . If the 



observation process satisfies Assumption 3.1 . then for any i > 1, k — 1, . . . ,m, 



(12) 



Proof. Throughout i > 1 is fixed and so are k,li, . . . ,li E {1, . . . , m}. Define the set 1; ~ {1, . . . , i} 

and let ga{x) := ag{ax). 

Conditioning on Wj — yj for j G li and Wi+i — x and using the independence assumption between the 
process {v4(t)} and the iid rvs {Wj)j with pdf g{-), gives 



P 



= / iP{A{{x,+x)/a)^Ak\A{xj/a)^Ai^J eI,)~ak)gix)dxYlg{yj)dyj 

= / iP{A{xi + x) = Ak\A{xj) Ai.,j e I,) - ak)ga{x)dxYl 9aiyj)dyj 

J[0,oo)' Jx=0 ^-gj^ 

= f [ {P{A{x)^Ak\A(xj-Xi) = Ai^,jch)~ak)ga{x)dx\lga{yj)dy^ (13) 



with Xj := X]z=i Vj^ J G ^i; where we have used the stationarity of the process {A{t)}^^ to derive (13) 
Define 

X . — {xi , . . . , ) , 

/(u,x*) := P{A{u) ^ Ak I A{xj - x,) = Ai^,j e h)) - cjk , 
F{x,x') := f f{u,^')du. 



Note that |/(u,x')| < 1 for any u,x* so that |F(a::,x')| < x for any x,x\ In this notation (131 rewrites 



P (A\f = Ak I Aj^; = Ai^.j G /,) - dfe = / / f{x,^')ga[x)dx TT ga[y,)dy,. (14) 



Integrating by parts and using the definition of gaix) and Assumption 3.1 yields 



f{s,x')gaix)dx = [gaix)F{x,ii')]^ ~ F{x,y.')g'^{x)dx 
Jo 

= \\m ga{x)F{x,:x}) — I F{x,jd)g' {ax)dx 

/>oo 

= -a^ / F{x,x')g'{ax)dx , 
Jo 



(15) 
(16) 
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where the hmit in (15) is zero since < ga{x)F(x,x^) < ga{x)x and hnij;_5.oo xg{x) = (see Assumption 



3.1). Combining (|14| and (|16|) gives 
P 



(^[;|^) = Ak I A\f^ = e /,) - ak ^-a" I / F{x,^')g' {ax)dx [] 5a(%)d2/,. (17) 

^ J[0,oo)» Ja;=0 ^gj. 



Fix e > 0. By Prop. |2.2| we know that there exists < Tg < oo, denoted as T from now on, such that for 
all X > T, \F{x, x*)/a;| < e/2, uniformly in x* or, equivalently, uniformly in . . . , G [0, oo). We have 



(Hint: use Assumption 3.1 and inequality |F(a;,x*)| < x) 

/•oo /"T pc 

/ F{x,x.')g'{ax)dx < F{x,x')g' {ax)dx ~ 

Jo Jo Jt 



/•oo 


F{x,yi') 


Jt 


x 



xg'{ax)dx 



< —T I g'{ax)dx 



xg'{ax)dx 



< — T J g'{ax)dx ~ 2 I xg'{ax)dx 

Tg{0) £ 

a 2a2 



(18) 



so that, from fL7|, 
P 



(aJ:!^) = Ak I a\]] = Ai^ ,jel?l~a,\ < (aTg(O) + j^^ J\ ga{y,)dVj 

= aTg(0) + ^. 



(19) 



We observe from ( 19 ) that 



< e 



□ 



for any < a < e/{2Tg{0)), which completes the proof since e is arbitrary. 
Application of the chain rule yields the following result. 

Proposition 3.1 As a — !■ the sequence {A|°j}j>i converges to an iid sequence with distribution 



It is now straightforward to describe the behavior of a constant rate walker. Take the observation 
process to be Poisson with rate 7' > 7 (recall that 7 is the walker rate of our constant rate walker), 
namely P{Wj < x) = 1 — e ' ^. Consider the dynamic graph {^f"' (t)}tgR associated with the RMPP 



(see (11)). We embed the times that the walker takes a step into the observation process. At 
each observation the walker takes a step with probability 7/7'; otherwise it does not with probability 
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(t' ■ Let Jjj"'' £ { 1 , . . . , 71} denote the position of the walker immediately after the j-th observation 

of the observation process {j > 1), and let 11^°"^ be the walker position at time t = Q. We assume that 
lim^-i-o P{uj^°''^ — v) exists for all w = 1, . . . , 71. This is the case, for instance, if f/g"'' is constant for any 
a > 0. 

The following equation describes the behavior of {v) = Vwia^Q P{U^'^^ = v), the fraction of observa- 
tions after which the walker resides at node v V. Assume first that this limit exists for any j > 1 and 
V = 1, . . . ,n. Let P :— [P(u, v)] be an n-by-n stochastic matrix with (w, i;)-entry given by 

P(«,«) = I =^ + ^Er=i<^a{rf,,„=o}, if"-^^, (20) 

[0, otherwise, 

u,v — 1, . . . ,n. We have 

n n i / A /' \ \ 

nfiv) = limY,Y.P (Al) - f^i-'i = X '^M > 0) + liu = - 0) ) 

n m / A ( \ 

= J2T. 1™ p (^S = ^0 1™ p (^i-^i = '"^ ^^^^P^m.u > 0) + i(u = v)i{dk,u = 0) 

a— >-0 \ / a— ^0 \ / \ (i'k.u 

u—1 k—1 ^ ' 

n 

= Y.V{u,v)^f\{u), (21) 

•u=l 

for J > 1 and u = 1, . . . , n, where d^.u is the degree of node u G V in graph configuration Ak and Ak{u, v) 
is the (u, w)-entry of the adjacency matrix (i.e. Aj^iuyv) = 1 if there is a link between vertices u and 



V in configuration k and zero otherwise). The second and third equalities follow from Proposition 3.1 



The existence of linia_>.o P{Uj°'^ = v) for all j > 1 and u = 1, . . . , 71 can be shown by induction on j based 
on 

Define Trj"-* :— (7r^°''(l), . . . , tt^'^' (n)). With these definitions (21) rewrites in the following matrix form 

vrf ^vrjV, j>l, 

with ttq^'' the n-dimensional vector where all entries are equal to zero except entry C/q"'' that is equal to 
1. To show that tt^"-* is unique, we prove that P is irreducible. The definition of T-connectivity and the 
fact that (Tfc > 0, k — 1, . . . ,m imply that P is an adjacency matrix of a strongly connected directed 
graph and strong connectivity of this graph is equivalent to the irreducibility of P [SJ Theorem 6.2.24], 
the details of this proof are found at Appendix |D] As P is also aperiodic, as the diagonal elements of P 
are non-zero, then the associated discrete-time Markov chain is ergodic since the state-space is finite so 
that, by Markov chain theory, the limit n^^^ = limj_>.oo tt^"'' exists and is given by the unique solution of 

n 

^(0)^^(0)p^ ^^W(u) = l. 
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Because of the PASTA property the steady state probabihty tt*^*^^ = Muij^ao T^f on the observation events 
is also the distribution in time, i.e., in the Hmit as a — > 0, tt = vr*^"^. 

The case where the walker rate depends on the node and graph configuration in which the walker resides 
yields a similar characterization. 



Proposition 3.2 Let "if be a stationary, ergodic, and T-connected dynamic graph and let {{A(t), U{t))}t>o 
be an associated CTRW with walker rates T = {Pk.v : fc = 1, . . . , m, u € V}. Let /^max > sup(r). Then, 
in the limit as a ^ 0, the stationary walker position distribution tt satisfies 

TT = ttP , 

where 

dk,u > 0, 

v^u (22) 
V = u . 



/3„ 



0ma 



The proof is similar to that given for the constant rate walker case and is omitted. The assumption 
that /3max > sup(r) ensures that P is aperiodic and the assumption of T-connectivity ensures that P is 
irreducible. 



Proposition 3.2 shows, as a — !■ 0, the steady state distribution of a walker with state dependent rates to 
be just a function of cr (the stationary distribution of {A(t)}f>o), the set of configurations A, and the 
walker rates /3, regardless of the graph dynamics. 



3.2 Time-scale Invariant Stationary 
Distribution 

In this section we turn our attention to a sufficient condition where the CTRW stationary distribution is 
invariant to the walker time scale 7. Consider a CTRW {{A{t), ?7(t))}t>o with non-zero walker rates on 
a stationary, ergodic, and T-connected graph . 

The key insight into our sufficient condition is the following: If there exists a tt that is the CTRW station- 
ary distribution given any (static) configuration Ai, . . . , A„i , then once the CTRW reaches distribution 
TT it remains with distribution tt independent of the graph dynamics. The key challenge is to show that 
the CTRW always converges to distribution tt, irrespective of the graph dynamics. We see that this is 
true if \1/ is stationary, ergodic, and T-connected. However, we also believe that the following results can 
be extended to some families of non-stationary dynamic graphs. 
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We first present the notation used in this section. It will be useful to describe the walker rates as a row 
vector 7(7) = {■j/3k,v)vev,k=i,...,m- 



The CTRW confined to a given configuration Ak is a Markov chain, k — 1, . . . ,m. Let 

Ak{i,jhl3k,i/dk,z, ^iiy^j, 
Y^jev Ak{i,j)lPk.i/dk,t, iii^ j , 

where Ak(i,j) is the element of Ak, be the infinitesimal generator of {U{t)}t>Q given configuration 

Assumption 3.2 (Fixed point tt*) Let A be the set of graph configurations of ^ and j be a set of 

walker rates such that there exists a tt* G [0, 1]", X^uev — 1j ^^'^^ ^ fixed point solution to 

= 7v*QiM,i) , yMeA. (23) 



In what follows we show that if A and 7 satisfy Assumption 3.2 then limf^oo ^[C^(^) — v] — tv*{v), 
\/v G V. Moreover, we show that tt* is independent of 7. We are now ready for the main result of this 
section. 

Theorem 3.4 Let graph process ^! be a stationary, ergodic, and T-connected graph with configuration 
set A. Let {{A(t), U{t))}t>o be a CTRW on the dynamic graph associated with 5". The CTRW 



has walker rates 7(7) — {'jPk,v}, w G V, A; = 1, . . . , m, 7 > 0. Lf A and 7(1) satisfy Assumption 3.2: then 



lim P[U{t) =v]= Tr*{v) , Vw e 



where tt* solves (23). Moreover, tt* does not depend on 7. 



Proof. For now assume 7=1. Let Ii{t) = {P[U{t) = w])t,gy. The Kolmogorov forward equation gives 

^=n(t)Q(A(t),7(l)). (24) 
From Assumption |3.2| there exists is that is a solution to ([23]). Hence, n(i) = tt* is also a solution 



to (24 1 where dll{t)/dt — 0. It follows from Theorem 3.1 that there is no other solution to (24) and 
therefore limt_i.oo P[U{t) — v] = 7r*(u), Vw G V. To show that tt* does not depend on 7, note that for 
any a > and k ~ 1, . . . ,m 

Oa = 7r*Q(Afc,7(l))a = 7r*Q(Afe, 7(a)). 

□ 



Examples of adjacency matrix sets A = {Ak : fc = 1, . . . ,m} that satisfy Assumption 3.2 for a constant 
rate walker, 7(7) = (7, . . . , 7) include: 
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• Regular graphs: Ai, i = l,...,m, consists of Ci > 1 connected components where the j-th 
connected component (j = 1, . . . , d) is a d^'''-regular graph (dj'"* 0). 

• Nodes V ^ V alternate between isolated and connected with constant degree, i.e., 

dk{v) G {0, d{v) > 0, k ~ 1, . . . ,m. Figure [l] ihustrates a dynamic graph that satisfies these 

requirements. 



Conditions imposed on the walker rates 7(7) can also guarantee that Assumption 3.2 is valid for any set 
of graph configurations A, as long as {A{t)}t^m. is T-connected. Consider the coupled CTRW and graph 
dynamics that satisfies Assumption |3.2| in the following proposition, stated without a proof: 



Proposition 3.3 (Degree proportional walker) Let A — {Ak : k ~ l,...,m} be a set of graph 
configurations. If the walker rates are 7(7) ~ {'ydk.v)v<£V.k=i,...,m, where dk,v is the degree of node v at 
configuration k. Then Theorem 3.4 is satisfied. Moreover, tt* — (i, . . . , ^). 



Proposition 3.3 has an interesting application. We can uniformly sample nodes (in time) without knowing 
the underlying topologies, A, or graph dynamics, {^(t)}tgR, as long as {A{t)}t^s. is stationary, ergodic, 
and T-connected. So far we have focused on conditions that allow us to obtain the stationary distribution 
of the walker. In what follows we present some case studies solved numerically. 




Figure 1 : Illustration of a dynamic graph whose node degrees are either kept constant in all graph configurations 
or there are isolated nodes. We make no assumption about the holding times of each graph configuration except 
that the graph is T-connected. 



4 Case Studies 



In previous sections we characterized the stationary behavior of an RW on a broad class of dynamic 
graph processes {{Xi, Si)}i>z- In this section, we focus mostly on random walks on Markovian dynamic 
graphs where {Xi}i>o forms a Markov chain and {Si}i>o is a sequence of independent and exponentially 
distributed random variables. Note that this model allows state dependent graph holding times to be 
taken into account in the Markov chain {Xi}i>o- In what follows we provide several examples and 
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numerical evaluations. Later in the section, we also show how the proposed framework can be applied to 
a Delay- Tolerant Network (DTN) scenario. 



4.1 Markovian dynamic graph examples 



In this section we present numerical results for some toy Markovian dynamic graph examples, that 
illustrate the non-trivial behavior of random walks on dynamic graphs and that support our theoretical 
findings. 

4.1.1 Star-circle example 




(a) The star-circle graph dynamics. 




-2-10 1 2 3 4 
Walker rate (10") 



(b) Steady state distribution of walker as a function of walker rate 
(nodes 5, 6 ,7 not shown for clarity). 



Figure 2: The star-circle graph dynamics and behavior of CTRW as a function of walker rate. 



We begin by considering a very simple model, consisting of just two graph snapshots: a star and a circle, 
as illustrated in Figure 2a The graph transits from one snapshot to another with rates A12 — A21 = 1. 
Thus, the average time in each graph is 1/2. Note however, that edges (1,2) and (1,10) are always present, 
since they exist in both configurations. 

We investigate the steady state solution of the CTRW on this dynamic graph. Figure [2b] shows the 
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steady state distribution of the random walk as a function of the walker rate (for clarity, only a subset 
of the states are shown). Note that the stationary distribution of the walker depends on the walker 
rate and converges to different distributions as the walker moves faster or slower. Indeed, the numerical 
results obtained are in agreement with the theoretical distributions for the fast and slow walker given in 
Sections |3 . 1 . 1 1 and |3. 1 . 2[ respectively. Finally, we note that in a graph with n nodes, node one alternates 
between degrees n ~ I and two. As n increases the dependence on the walker speed is magnified. 

4.2 Edge Markovian model and examples 

In this section we consider some particularities of random walks on a special class of dynamic graphs 
called edge Markovian graphs. Start with a fixed adjacency matrix A and attach an independent On-Off 
process to each edge with exponentially distributed holding times. In edge Markovian graphs, edges 
alternate between being present and absent from the graph according to independent On-Off processes. 
Let E be the set of edges in the graph described by A. Let Ao(e) and Ai(e) denote the rate at which 
edge e G E changes from the On to the Off state and from the Off to the On state, respectively, which 
can vary from edge to edge. 

A few observations on the model follows. Let Ak be a particular configuration of the dynamic graph model. 
In particular, the edge Markovian model induces a total of m = 21-^1 configurations, which represent all 
possible labeled subgraphs over an edge set with \E\ edges. Moreover, consider any transition between 
two configurations induced by the model. The graphs corresponding to these two configurations differ by 
exactly one edge, since the on-off processes associated with the edges are continuous in time. Moreover, 
the rate associated with this graph transition is given by the corresponding rate of the edge (either its 
On rate or Off rate). 

Let Quv denote the stationary fraction of time that edge (u, v) E is On, which is simply given by 

Quv = Ai(u,w)/(Ao(u, w) + Ai{u,v)). 

Let cTfc denote the steady state fraction of time that the edge Markovian model spends in configuration 
Ak, k — 1, . . . , 2l^l. In particular, we have 

CTfc = H^k{u,v)}quv + - 'i-{Ak{u,v)}){l - Quv) ("25) 

{u,v)€E 

Note that at is given by the product of the probabilities the egdes that define Ak are present while all 
other edges are absent. Since all edges are independent, this is trivially obtained as shown above. Edge 
Markovian graphs are of interest due to their simple description and structure. However, as we will soon 
see in our numerical results, the steady state distribution of a constant rate random walk on this model 
depends on the walker rate. It is an open problem whether the walker steady state distribution can be 
obtained in closed form. 
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4.2.1 Edge Markovian: n = 3 



Consider the edge Markovian graph model over a complete graph with 3 nodes. The number of different 
configurations is 2^ = 8, out of which 4 have at least one isolated vertex (graph not connected). Moreover, 
let Ai(l,2) = Ao(l,2) = 10'', Ai(l,3) = Ao(l,3) = 1, Ai(2,3) = Ao(2,3) = 1. Thus, pe = 0.5 for every 
edge (i.e., all edges have the same time average), and thus, all configurations have the same time average 



probability of 1/8, as given by equation (25) 



Figure [3a] shows the exact steady state distribution of the random walk as a function of the walker 
rate. Interestingly, while all edges have the same time average and all configurations Ak, k — 1, . . . , 2l^l, 
have the same ak, the walker steady state distribution still depends on the walker rate. Moreover, the 
behaviors of the fast and slow walkers differ. While the slow walker converges to a uniform distribution 
over the nodes, the fast walker always favors node 3, the node not incident to the fastest changing edge 
(1,2). Despite the relatively small differences in the walker distribution (P[l/l^ = 3] varies by 7%), the 
point of this example is to illustrate that such differences can arise even in small and simple models. 
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Figure 3: Characteristics of random walks on a edge Markovian graph model. 

Figure |3b] also shows the maximum absolute difference between our theoretical results for the fast and 
slow walker and the actual distribution obtained exactly. In particular, we present the total variation 
distance between the two distributions, defined as maxi=i....^„ K(*) ~ '"''(Ol; where tt is the exact walker 
distribution and tt' is either the fast or slow walker distribution, and n is the number of nodes in the 
graph. For walker rates greater than one, the theoretical results for the fast walker were used (tt' as 



defined in Section 3.1.1 1, while for rates smaller than one the results for the slow walker were used (tt' as 



defined in Section 3.1.2). Note that as the walker slows down or speeds up, the total variation decreases 



20 



(graph in log- log scale). Our numerical results indicate that the fast and slow walker become close to our 
asymptotic results as their rates increase and decrease, respectively. When the timescales of the walker 
and graph dynamics are similar, the total variation distance between the asymptotic cases and the steady 
state distribution is relatively larger, as expected. 

Moreover, if we consider the "degree-time distribution" of a given node, namely, the fraction of time 
that node v £ V has degree 0, 1, . . ., we note that all three nodes have identical degree-time distributions 
but the fraction of time the walker spends on each node varies. This indicates that the degree-time 
distribution is insufhcient to characterize the walker steady state. This is also like true for transient 
metrics as well. 



4.2.2 Edge Markovian: 6-node kite 



Now consider an edge Markovian process over the "kite graph" illustrated in Figure 4a In particular, let 
all thin edges e have On-Off rates Ai(e) = Ao(e) = 1. Similarly, let all thick edges e' have On-Off rates 
Ai(e') — 100, — Ao(e') — 10. Note that locally all nodes are connected through identical and independent 
On-Off processes, two thin edges (On-Off rates equal to one) and one thick edge (On-Off rates equal to 
one hundred). Thus, in some sense, "locally" all nodes are indistinguishable. Surprisingly, even in this 



case the behavior of the walker depends on its rate, as shown in Figure 4b Clearly, the structure of 
the graph plays an important role, as illustrated in this example, as the fast and the slow walkers have 
different time stationary distributions for different nodes. This indicates the difficulty of characterizing 
the exact behavior of random walks in general graph dynamics, even if we limit ourselves to the class of 
edge Markovian graphs. 



4.3 A simple vehicular DTN 



In this section we consider an application of our modeling framework to a simple vehicular disruption- 
tolerant networks (DTN) model. Our model captures some essential characteristics of DTNs. Consider a 
set of buses equipped with wireless routers moving around according to their routes. Two buses establish 
communication when they are within the coverage radius of the wireless routers. Buses belonging to the 
same line (route) move from one bus stop to another following a predefined sequence of stops in a circular 
fashion. The following notation is used to describe the model: S = {si, . . . , So} is the set of bus stops 
across all bus lines, where o is the number of different stops; Li e S"^ is a vector with the sequence of 
stops for bus line i, and Ui is the number of stops at bus line i; I denotes the number of different bus lines 
and bi is the number of different buses operating in line i — 1, . . . ,1; Assume that both the amount 
of time line i bus stays at stop fc, and C^;, the amount of time it takes line i bus to move from stop k to 
I, are exponentially distributed random variables, but can have different parameter values for any i,k,l. 
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(a) The edge Markovian model over the 
6-node kite graph where edges belong to 
one of two On-Off processes. 



(b) Steady state distribution of walker as a function of 
walker rate (nodes 5 and 6 not shown, but are identical to 
nodes 2 and 3). 



Figure 4: Characteristics of random walks on a 6-node kite edge Markovian graph model. 



In addition, buses move independently of each other, including those in the same line. 

The bus routes and the coverage radius of the wireless router allow two or more buses to exchange 
information when buses are at the same stop. Thus, if two or more different bus lines share at least 
one bus stop in their route, then buses from these lines will be able to communicate at the shared bus 
stops. Moreover, two or more buses from the same line can communicate in any stop of their line, as 
they can always meet at these stops. Finally, we assume that the communication radius is smaller than 
the distance between bus stops, such that buses only communicate when located at a shared bus stop. 



Figure 5a shows an example with 11 bus stops, three bus lines {I = 3), defined by Li = (si, S2, S3, S4, S5), 
L2 = {ss, S4, se, Sr, Ss) and L3 = (57, sg, sio, Sn), and four buses: 61 = 1, 62 = 1, &3 = 2 (line three has 
two buses). Note that lines 1 and 2 share two bus stops (ss and S4) and that lines 2 and 3 share one bus 

stop (sy). 

Consider a continuous-time random walker (CTRW) moving around the buses with rate 7. The goal is 
to determine the fraction of time that the walker spends in each bus or in each bus line. This problem 
can be formulated and solved using the modeling framework proposed in this paper. The first step is 
to construct a dynamic connectivity graph model from the movement of the buses in their respective 
lines. In particular, each bus is a node in the graph, since this corresponds to a possible location for the 
random walk. Moreover, each possible configuration of buses on their stops will define a connectivity 
graph, where nodes (buses) in the same stop are all within communication radius of one another. Note 
that each connectivity graph is composed of connected components that are all cliques (fully connected 
subgraph), since all buses in the same stop can communicate. 
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(a) Example of a vehicular DTN with eleven bus stops, (b) Connectivity graphs and transitions between them 
three bus lines and four buses (line 3 has 2 buses). according to the bus system illustrated in Figure [5a[ 

Figure 5: Example of a simple bus system (a) and the induced dynamic graph model (b). 



Consider the example in Figure 5a and the possible connectivity graphs that can be created, which are 
illustrated in Figure 5b The connectivity graph has four nodes, corresponding to the four buses. Each 
bus can be in a different stop, thus yielding a connectivity graph with no edges. Also, bus in line 2 can be 
at stop S7 at the same time as the two buses from line 3, yielding a connectivity graph where these three 
buses are all connected. Finally, note that not all graphs with four nodes are possible, since different 
lines may not have stops in common such as lines 1 and 3, for instance. 



The transitions between graph configurations are shown in Figure 5b In our model, buses cannot 
simultaneously leave a stop. As a consequence, the number of allowed transitions is reduced. Once the 
dynamic graph model is constructed, we can obtain the state holding times for each graph configuration. 
This is possible if the holding times at bus stations and the amount of time it takes a bus to move from 
a station to another are exponentially distributed. However, this is non-trivial in the general case, since 
buses can move along their routes without changing the connectivity graph. Moreover, totally different 
bus configurations over the set of stops can lead to the same connectivity graph. Since our modeling 
framework makes no assumption on the state (static graph) holding times of the dynamic graph model 
we can extend the exponential assumption considering general distributions for the holding times at each 
graph configuration, assuming only that the expected holding time is finite for all static graphs and that 
dynamic graph process is stationary, ergodic and T-connected. 



The model constructed from this scenario matches the case studied in Section 3.2 in which the stationary 



distribution of the random walk is time-scale invariant and uniform over the set of nodes in the graph, 



independent of graph dynamics (see Theorem 3.4). This occurs since every connected component of every 
possible graph configuration is a clique, thus, having identical degree within each component. Therefore, 
the fraction of time the walker spends in any given bus is simply I / bj , while the fraction of time 
spent in bus line i is simply bi/ bj. Thus, CTRWs could find applications in searching for information 
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or sampling properties in such systems, as its steady state does not depend on the graph dynamics. 



5 Related work 

Random walks have been widely used to understand and characterize graphs due to their well understood 
steady state behavior. By leveraging the steady state distribution of random walks, principled mechanisms 
for characterizing and estimating vertex-related properties have been devised [H [Til 1131 HSl [16] . 

It follows that random walks can potentially be used to understand and characterize dynamic graphs. 
In fact, efforts in this direction concerning time-independent dynamic graphs (i.e., each snapshot is 
independent of the previous) have appeared in the literature [H [3 [HI [2], mainly in the context of 
determining upper and lower bounds for the cover time of random walks. More recently, proposals to 
define time-dependent dynamic graph models as well as characterize random walks in them have also 
appeared in the literature [ll[2l[3]. However, these efforts have focused on discrete-time dynamic graph 
models with a goal of computing the cover time of random walks either in special graph structures , 
in specific dynamic graph models or through numerical evaluations [IL Our work differs from these 
in the sense that we consider continuous-time dynamic graph models and continuous-time random walks 
with the goal of analytically characterizing the steady state behavior of the walker. Moreover, our prior 
work on this topic considered only Markovian dynamics and characterized the steady state behavior of 
the walker only under time-scale separation |17j . 

Finally, random walks have also been used as a sampling mechanism to estimate characteristics of vertices 
(e.g., fraction of vertices of a particular kind) in large static graphs 0[TTl[Tn]. More recently, efforts 
to measure characteristics of vertices in dynamic graphs have also appeared in the literature [151 120j . 
However, these are mostly preliminary and exploratory papers, indicating potential pitfalls and biases 
introduced by fast changing dynamic graphs. In contrast, our works is a first step at providing a theoretical 
foundation that can then be applied to estimate characteristics of vertices in dynamic graphs. 

6 Conclusion 

Understanding the long-term behavior of CTRW over dynamic graphs is an important step towards a 
comprehensive study of dynamic graphs. Since the steady state distribution of random walks on static 
graphs is arguably the most important characteristic of these processes, it is of vital importance to 
characterize the CTRW steady state distribution for a broad class of dynamic graph processes. Unlike 
random walks on static graphs, CTRWs on dynamic graphs have a non-trivial behavior. The walker 
rate as well as the process that governs the graph dynamics both impact the asymptotic time stationary 
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distribution of the walker (the amount of time a walker spends on each node). 

Our main results assume the graph process to be asymptotically stationary, ergodic and T-connected. 
We make no independence assumptions concerning the times the process resides in each of the graph 
configurations or the particular distribution of these residence times. In other words, the time spent 
in a given graph configuration can be dependent on the residence time at other configurations and, in 
addition, the distribution of the time the network spends in a configuration can be general. Moreover, the 
dynamic graph can have temporarily disconnected nodes as long as the dynamic graph is T-connected. 

Under this general scenario, we have obtained the steady state distribution of cases in which the walker 
is either much faster or much slower as compared to the rate of changes in graph configurations. We 
have obtained a sufficient condition for the CTRW stationary distribution to be invariant to the walker 
rate and presented examples that illustrate models in which these results are applicable. In this context, 
additional application examples for the constant degree constraint can be found in P2P networks where 
peers maintain a nearly fixed number of connections. Hence, Theorem |3.4| helps explain why sampling 
these dynamic networks using random walks leads to meaningful results |20| . 

The examples in this paper serve mainly for illustrative purposes. However, we believe that the theory 
we developed will find applications in many important areas. Examples of promising application areas 
are in sampling DTN and P2P networks. 
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Appendices 



A Proof of Proposition 2.1 



From a classical construction of ergodic theory (see e.g. [TSl Prop. 11.4]), we know that there exist a 
probability space (fi, J", P) and ergodic endomorphisms {9*)teR, 0* : — >■ f2 - called a flow - on this 
probability space on which the stationary sequence — {Xn, Sn}nei, is defined, such that 

Sn^S o e'\ X^ = X o r, n e Z, (26) 

with (5",^) rvs distributed as (5„,X„) and § -.^ 9^° . 



Observe that the stationary sequence defined in ( 26 ) satisfies 



N N 

J™ M E /(^"' = j™ M E /(^' ^) o = ^[/(^' ^)] a.s. (27) 

TV— ^oo iV ^ — ' N^oo I\ ^ — ' 

n—l n—l 
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for any nonnegative measurable mapping / on ft, that is, it is an ergodic sequence. The second equahty 
in (27) follows from the ergodicity of the flow (6'*)tgR. 



Furthermore, for any n G Z, cj G fJ, 

T„(rH) = r„+feH-s, (28) 



where k is the unique integer such that Tk{uj) < s < Tk+i{io)^A more compact notation for (28) is 
T^oO'' = Tn+k - s if Tfc < s < Tfc+i. 

We are now in position to prove the stationarity of {A(t)}tgR. 
Recafl that A{t) = J2nez ^nl(r„ < i < T„+i). We have 

A{t) o9' = ^nHT„ 6' <t< Tn+i o r). 



IfTfc<s<rfe+i,by (281 



A{t)oe' = ^X„l(r„+fe <t + s<T„+i+fe) 
nez 

= ^X„_fel(T„ <i + s <r„+i) 

= e'^" ^nliTn < t + S < Tn+l) 

nez 

= e'^" o A{t + s), 

where we have used the relation Xn-k = Xn o 6^'=. The above can be rewritten as 

A{t) o r = A(i + s) o J2 S^'HTk < s < Tk+i) 

kez 

or, equivalently, 

A{t + s) = A{t) o J2 S^'^'MTk < s < Tfe+i). 
kez 



^In words, j28| says that for ui = (sn, s„)„^x S the nth point of the point process (tn)n^z (with — tn = Sn) 

to the right (resp. left) of t = when the trajectory is shifted by s (i.e. x„ is replaced hy Xn — s for every n S Z) is the 
(n + fc)-th point to the right (resp. left) of i = if tfc < s < tj,. 
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This shows that {A{t)}t is stationary since 



P{A{t + s)eC) = 



In particular, 



p [ ^ A(t) o e^-^-iin < s < n+i) e c] 

\kez J 

= ^ P{A{t) o r-^" e C, l{n < s < Tfc+i)) 
kez 

= ^P(A(t) GC,l(Tfc <5<r,+i)) 



kez 

P{A{t) e C). 



A{s) = A(0) o e'-^n{Tk < s < Tk+i). 



kez 



B Proof of Proposition 2.2 



Fix e > 0. All rvs are defined on the probability space {fl,J-,P). By Egorov's theorenj^ TIT, pg. 43, 
Theorem 2] we know that there exists a Borel set C J- with P{Ce) < e, such that the convergence 
in Theorem |2.1| is uniform on il — C^, namely, there exists such that (we write A{x,ijj) for A{x) to 
emphasize that A(x) is a P-measurable rv) 



1 f* 

crk-e< - / l{A{x,uj)=Au:}dx < ak + e 
^ Jo 



for all t > and for all cj e - C<: . 



Let B e T with P{B) > 0. Integrating w.r.t. dP{uj) for it; e B - in (29) gives 

{ak-e)P{B-C,)< f dP{^)- j l{Ai..u.))=A,}dx < {uk + e)P{B - C,) 

JB-C, ^ Jo 



for all t > Tc and for all cj e S - Ce 
By Fubini's theorem 



/ / '^{A(x,^)=Ak}dxdP{uj) = iu>eB-cA{A(x,u,)=A^}dP{ 
JB-Ct JO JO Jn 



uj)dx 



P{A{x) =Akr\ {B-C,))dx. 



(29) 



(30) 



(31) 



^Egorov's theorem applies here since has a finite measure w.r.t. P as P{f2) = 1. 
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Since 



P{A{x) ^AkDB)- P{A(x) = Akn C,) 
P{A{x) = Ak\B)P{B) - P{A{x) = Ak\C,)P{C,), 



we obtain from (30 1- (32 1 that 



K-e)P(S-Ce) + P(C,)/(t,e) < ^ f P{A{x)) = Ak\B)d. 

Jo 



< K + e)P(6-C,) + P(C,)/(t,e) 
for all t > T„ with /(<, e) := (1/t) /J P{A{x) = Ak\C,)dx. 



Dividing (33) by P{B) (recall that P{B) > 0) gives 



(CTfe - e) 



P(B-C,) , P(C, 



PiB) P{B) 



1 

< t 



P{A{x) = Ak\B)dx 



< {(Jk + e) 



P{B-C,) ^ P(Ce) 



As e ^ 0, (i) ^/(t,e) ^ since < ^f{t,e) < 



(32) 



(33) 



(34) 



P{B) P{B) 

from the definition of and the fact 



that < f{t,e) < 1. On the other hand, (ii) both terms {at ± £)^^^^gy 



p{B) J ' ' ^ p(e) 

go to CTfe as e — > since 

lim.^o P{B - C,)/P{B) = lime^o(^'(S) - P{C^))/P{B) = 1. Since ^ c» as e we conclude from 
(i)-(ii) above and from (34) that lini4^oo(l/^) /g P{A{x) = Ak)\B)dx = cTfc for all Borel sets B such that 
P{B) > 0. I 



C Proof: T is finite 



The graph process {A{t)}t£K is time stationary and ergodic. It follows from Proposition 2.2 that there 
exists Tfc > s.t. 

\f l{A{s)= Ak)ds> ak/2, x>0,t>Tk 
independent of x. Choose /cq — argminj,{CTfc} and T' = max{Tfc}. Thus 

— / l{A{s) = Ak)ds>akj2, x>0. 

Consequently the graph process spends at least a}^^T' jl units of time in configuration fc during interval 
+ T') regardless of the state of the graph process aX t = x. Now consider the situation where 
J7i(a;) — u, U2{x) = w, and A{x) — A^- Let Ak — Ai^,Ai^, . . . ,Ai. be a sequence of graphs such that 
there is a temporal path between u in A;^ and w in Ai.. This requires that there be paths within each 
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graph configuration, pi = {v.ifl,Vi^i, . . . , Wi,„ J that satisfy vi^ = u, Wi+i,o = f j,n,, « = 1, • ■ • , j - 1, and 
^i,rij = u). Let /lu.iu denote the number of physical hops on this path and j = H^^w the number of 
configurations in the sequence. We focus now on the event that walker 1 progresses from node u to node 
w in the interval [x, x + jT') jT' by progressing across path pi during [x + {i~ 1)7"', x + iT') while walker 
2 remains at node w. The probability of this event, Pu,w is bounded from below by 

X (7^)'"" ri ^( E ^^'"^ < ^''^^■0/2 < E ^^-^) 

Here H^i^^ = max^u.w Hu^w, ^max = max^^u, and Z^^^, denotes the time between a walker arriving 
to node v in configuration Ai. and taking its next step. This time is exponentially distributed with rate 
'Yi^^v and there exists some q > such that 

rii — l Tii 

P{ Z^,,, < T'(Jkj2 < Z^^v,) > q 

for all u,w. Hence 

Pu,w>Po^e (- j q 

^max 

Last, To = F„,axT'. 



D Irreducibility of P 



Consider P as defined in ( 20 ) to be the adjacency matrix of a weighted directed graph G with self-edges. 
Then by [HI Theorem 6.2.24] P is irreducible if G is strongly connected (as the elements of P are finite, 
i.e., ||P||oo < 00). What we need to show is that G is strongly connected. Let 

7 . / cTfe 

e — — mm - — 

7' fc = l,...,m, Vney \dk.u 



where 7 and 7' as defined in (20 1. Clearly e > 0. Now decompose P into two parts: P = eX]fc=i ^fc + 



From the definition of P, (20 1, it is clear that S > 0. By the definition of T-connectivity (Definition 2.2) 
the (undirected weighted) graph with adjacency matrix A = X^feLi ^fc is connected. Thus, the graph 
with adjacency matrix A^ — eX^fcLi (remember that e > 0) must also be connected. As the adjacency 
matrix of G can be written as A,; + S, S > 0, then G is strongly connected, finishing our proof. 
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